Validating Distributed System Test Execution Correctness via System Execution Traces

Effective validation of quality-of-service (QoS) properties (such as event prioritization, latency, and throughput) in distributed real-time and embedded (DRE) system requires evaluating system capabilities in representative execution environments. This validation process typically involves executing DRE systems composed of many software components on many hardware components connected via networks. Unfortunately, evaluating the correctness of such tests is hard since it requires validating many states dispersed across many hardware/software components. This paper provides two contributions to research on validating DRE system capabilities and QoS properties. First, it presents the Test Execution (TE) Score, which a methodology for validating execution correctness of DRE system tests. Second, it empirically evaluates TE Score in the context of a representative DRE system. Results from this evaluation show that TE Score can determine the percentage correctness in test execution—thereby increasing confidence in QoS assurance and improving test quality.